Cross-document relationship classification for text summarization

Multiple documents describing the same event present some interesting challenges for natural language processing. They contain similar information and yet they also exhibit a number of interesting properties: paraphrases, partial agreement, difference in judgment and emphasis, and contradictions. When the sources track an event that evolves over time, more phenomena can be observed: additions, updates, corrections, comments, clarifications, etc. In the current work, we introduce CST (cross-document structure theory), a paradigm for multi-document analysis. CST takes into account the rhetorical structure of clusters of related textual documents. The goals of the paper are fourfold: first, we present a taxonomy of 18 CST relationships and give our motivation for proposing the theory. Second, we present an experiment that demonstrates that CST is essential for extractive, multi-document summarization. We then argue that CST can be the basis for multi-document summarization guided by user preferences for summary length, information provenance, cross-source agreement, and chronological ordering of facts. In particular, we propose a CST-enhancement procedure for extractive summarization, and demonstrate that CST-enhanced summaries outperform their unmodified counterparts as measured by the relative utility evaluation metric. Next we discuss the building of the CSTBank, a corpus of document clusters in which sentence pairs have been annotated for CST relationships. Finally, we discuss our efforts in the automatic detection of CST relationships between sentence pairs extracted from related documents. We show that a binary classifier for determining existence of structural relationships significantly outperforms the baseline. We also achieve promising results on the multi-class case in which the full taxonomy of relationships are considered.